Empirical and Gaussian processes on Besov classes

We give several conditions for pregaussianity of norm balls of Besov spaces defined over $\mathbb{R}^d$ by exploiting results in Haroske and Triebel (2005). Furthermore, complementing sufficient conditions in Nickl and P\"{o}tscher (2005), we give necessary conditions on the parameters of the Besov space to obtain the Donsker property of such balls. For certain parameter combinations Besov balls are shown to be pregaussian but not Donsker.Comment: Published at http://dx.doi.org/10.1214/074921706000000842 in the IMS Lecture Notes Monograph Series (http://www.imstat.org/publications/lecnotes.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures

We continue the investigation of Bernstein-von Mises theorems for nonparametric Bayes procedures from [Ann. Statist. 41 (2013) 1999-2028]. We introduce multiscale spaces on which nonparametric priors and posteriors are naturally defined, and prove Bernstein-von Mises theorems for a variety of priors in the setting of Gaussian nonparametric regression and in the i.i.d. sampling model. From these results we deduce several applications where posterior-based inference coincides with efficient frequentist procedures, including Donsker- and Kolmogorov-Smirnov theorems for the random posterior cumulative distribution functions. We also show that multiscale posterior credible bands for the regression or density function are optimal frequentist confidence bands.Comment: Published in at http://dx.doi.org/10.1214/14-AOS1246 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

A sharp adaptive confidence ball for self-similar functions

In the nonparametric Gaussian sequence space model an $\ell^2$-confidence ball $C_n$ is constructed that adapts to unknown smoothness and Sobolev-norm of the infinite-dimensional parameter to be estimated. The confidence ball has exact and honest asymptotic coverage over appropriately defined `self-similar' parameter spaces. It is shown by information-theoretic methods that this `self-similarity' condition is weakest possible.Comment: To appear in Stochastic Processes and Applications (memorial issue for E. Gin\'e

Confidence bands in density estimation

Given a sample from some unknown continuous density $f:\mathbb{R}\to\mathbb{R}$, we construct adaptive confidence bands that are honest for all densities in a "generic" subset of the union of $t$-H\"older balls, $0<t\le r$, where $r$ is a fixed but arbitrary integer. The exceptional ("nongeneric") set of densities for which our results do not hold is shown to be nowhere dense in the relevant H\"older-norm topologies. In the course of the proofs we also obtain limit theorems for maxima of linear wavelet and kernel density estimators, which are of independent interest.Comment: Published in at http://dx.doi.org/10.1214/09-AOS738 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Uniform limit theorems for wavelet density estimators

Let $p_n(y)=\sum_k\hat{\alpha}_k\phi(y-k)+\sum_{l=0}^{j_n-1}\sum_k\hat {\beta}_{lk}2^{l/2}\psi(2^ly-k)$ be the linear wavelet density estimator, where $\phi$, $\psi$ are a father and a mother wavelet (with compact support), $\hat{\alpha}_k$, $\hat{\beta}_{lk}$ are the empirical wavelet coefficients based on an i.i.d. sample of random variables distributed according to a density $p_0$ on $\mathbb{R}$, and $j_n\in\mathbb{Z}$, $j_n\nearrow\infty$. Several uniform limit theorems are proved: First, the almost sure rate of convergence of $\sup_{y\in\mathbb{R}}|p_n(y)-Ep_n(y)|$ is obtained, and a law of the logarithm for a suitably scaled version of this quantity is established. This implies that $\sup_{y\in\mathbb{R}}|p_n(y)-p_0(y)|$ attains the optimal almost sure rate of convergence for estimating $p_0$, if $j_n$ is suitably chosen. Second, a uniform central limit theorem as well as strong invariance principles for the distribution function of $p_n$, that is, for the stochastic processes $\sqrt{n}(F_n ^W(s)-F(s))=\sqrt{n}\int_{-\infty}^s(p_n-p_0),s\in\mathbb{R}$, are proved; and more generally, uniform central limit theorems for the processes $\sqrt{n}\int(p_n-p_0)f$, $f\in\mathcal{F}$, for other Donsker classes $\mathcal{F}$ of interest are considered. As a statistical application, it is shown that essentially the same limit theorems can be obtained for the hard thresholding wavelet estimator introduced by Donoho et al. [Ann. Statist. 24 (1996) 508--539].Comment: Published in at http://dx.doi.org/10.1214/08-AOP447 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

Adaptive estimation of a distribution function and its density in sup-norm loss by wavelet and spline projections

Given an i.i.d. sample from a distribution $F$ on $\mathbb{R}$ with uniformly continuous density $p_0$, purely data-driven estimators are constructed that efficiently estimate $F$ in sup-norm loss and simultaneously estimate $p_0$ at the best possible rate of convergence over H\"older balls, also in sup-norm loss. The estimators are obtained by applying a model selection procedure close to Lepski's method with random thresholds to projections of the empirical measure onto spaces spanned by wavelets or $B$-splines. The random thresholds are based on suprema of Rademacher processes indexed by wavelet or spline projection kernels. This requires Bernstein-type analogs of the inequalities in Koltchinskii [Ann. Statist. 34 (2006) 2593-2656] for the deviation of suprema of empirical processes from their Rademacher symmetrizations.Comment: Published in at http://dx.doi.org/10.3150/09-BEJ239 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions

The problem of determining a periodic Lipschitz vector field $b=(b_1, \dots, b_d)$ from an observed trajectory of the solution $(X_t: 0 \le t \le T)$ of the multi-dimensional stochastic differential equation \begin{equation*} dX_t = b(X_t)dt + dW_t, \quad t \geq 0, \end{equation*} where $W_t$ is a standard $d$-dimensional Brownian motion, is considered. Convergence rates of a penalised least squares estimator, which equals the maximum a posteriori (MAP) estimate corresponding to a high-dimensional Gaussian product prior, are derived. These results are deduced from corresponding contraction rates for the associated posterior distributions. The rates obtained are optimal up to log-factors in $L^2$-loss in any dimension, and also for supremum norm loss when $d \le 4$. Further, when $d \le 3$, nonparametric Bernstein-von Mises theorems are proved for the posterior distributions of $b$. From this we deduce functional central limit theorems for the implied estimators of the invariant measure $\mu_b$. The limiting Gaussian process distributions have a covariance structure that is asymptotically optimal from an information-theoretic point of view.Comment: 55 pages, to appear in the Annals of Statistic